(1) Field of the Invention
The invention generally relates to signal processing/data processing systems for processing time series distributions containing a small number of data points (e.g., less than about ten (10) to twenty-five (25) data points). More particularly, the invention relates to a two-stage system for classifying the white noise degree (randomness) of a selected signal structure comprising a time series distribution composed of a highly sparse data set. As used herein, the term “random” (or “randomness”) is defined in terms of a “random process” as measured by a selected probability distribution model. Thus, pure randomness, pragmatically speaking, is herein considered to be a time series distribution for which no function, mapping or relation can be constituted that provides meaningful insight into the underlying structure of the distribution, but which at the same time is not chaos.
(2) Description of the Prior Art
Recent research has revealed a critical need for highly sparse data set time distribution analysis methods and apparatus separate and apart from those adapted for treating large sample distributions. This is particularly the case in applications such as naval sonar systems which require that input time series signal distributions be classified according to their structure, i.e., periodic, transient, random or chaotic. It is well known that large sample methods often fail when applied to small sample distributions, but that the same is not necessarily true for small sample methods applied to large data sets.
Very small data set distributions may be defined as those with less than about ten (10) to twenty-five (25) measurement (data) points. Such data sets can be analyzed mathematically with certain nonparametric discrete probability distributions, as opposed to large-sample methods which normally employ continuous probability distributions (such as the Gaussian).
The probability theory discussed herein and utilized by the present invention is well known. It may be found, for example, in works such as P. J. Hoel et al., Introduction to the Theory of Probability, Houghton-Mifflin, Boston, Mass., 1971, which is hereby incorporated herein by reference.
Also, as will appear more fully below, it has been found to be important to treat white noise signals themselves as the time series signal distribution to be analyzed, and to identify the characteristics of that distribution separately. This aids in the detection and appropriate processing of received signals in numerous data acquisition contexts, not the least of which include naval sonar applications. Accordingly, it will be understood that prior analysis methods and apparatus analyze received time series data distributions from the point of view of attempting to find patterns or some other type of correlated data therein. Once such a pattern or correlation is located, the remainder of the distribution is simply discarded as being noise. It is believed that the present invention will be useful in enhancing the sensitivity of present analysis methods, as well as being useful on its own.
Various aspects related to the present invention are discussed in the following exemplary patents:
U.S. Pat. No. 6,068,659, issued May 30, 2000, to Francis J. O'Brien, Jr., discloses a method for measuring and recording the relative degree of pical density, congestion, or crowding of objects dispersed in a three-dimensional space. A Population Density Index is obtained for the actual conditions of the objects within the space as determined from measurements taken of the objects. The Population Density Index is compared with values considered as minimum and maximum bounds, respectively, for the Population Density Index values. The objects within the space are then repositioned to optimize the Population Density Index, thus optimizing the layout of objects within the space.
U.S. Pat. No. 5,506,817, issued Apr. 9, 1996, to Francis J. O'Brien, Jr., discloses an adaptive statistical filter system for receiving a data stream comprising a series of data values from a sensor associated with successive points in time. Each data value includes a data component representative of the motion of a target and a noise component, with the noise components of data values associated with proximate points in time being correlated. The adaptive statistical filter system includes a prewhitener, a plurality of statistical filters of different orders, stochastic decorrelator and a selector. The prewhitener generates a corrected data stream comprising corrected data values, each including a data component and a time-correlated noise component. The plural statistical filters receive the corrected data stream and generate coefficient values to fit the corrected data stream to a polynomial of corresponding order and fit values representative of the degree of fit of corrected data stream to the polynomial. The stochastic decorrelator uses a spatial Poisson process statistical significance test to determine whether the fit values are correlated. If the test indicates the fit values are not randomly distributed, it generates decorrelated fit values using an autoregressive moving average methodology which assesses the noise components of the statistical filter. The selector receives the decorrelated fit values and coefficient values from the plural statistical filters and selects coefficient values from one of the filters in response to the decorrelated fit values. The coefficient values are coupled to a target motion analysis module which determines position and velocity of a target.
U.S. Pat. No. 6,466,516 B1, issued Oct. 15, 2002, to Francis J. O'Brien, Jr. et al., discloses a method and apparatus for automatically characterizing the spatial arrangement among the data points of a three-dimensional time series distribution in a data processing system wherein the classification of said time series distribution is required. The method and apparatus utilize grids in Cartesian coordinates to determine (1) the number of cubes in the grids containing at least one input data point of the time series distribution; (2) the expected number of cubes which would contain at least one data point in a random distribution in said grids; and (3) an upper and lower probability of false alarm above and below said expected value utilizing a discrete binomial probability relationship in order to analyze the randomness characteristic of the input time series distribution. A labeling device also is provided to label the time series distribution as either random or nonrandom, and/or random or nonrandom within what probability, prior to its output from the invention to the remainder of the data processing system for further analysis.
U.S. Pat. No. 6,397,234 B1, issued May 28, 2002, to Francis J. O'Brien, Jr. et. al., discloses a method and apparatus for automatically characterizing the spatial arrangement among the data points of a time series distribution in a data processing system wherein the classification of said time series distribution is required. The method and apparatus utilize a grid in Cartesian coordinates to determine (1) the number of cells in the grid containing at least-one input data point of the time series distribution; (2) the expected number of cells which would contain at least one data point in a random distribution in said grid; and (3) an upper and lower probability of false alarm above and below said expected value utilizing a discrete binomial probability relationship in order to analyze the randomness characteristic of the input time series distribution. A labeling device also is provided to label the time series distribution as either random or nonrandom, and/or random or nonrandom.
(3) Description of Another Department of the Navy Developments
A development in a related technological area made by the U.S. Department of the Navy is described in U.S. Pat. No. 6,597,634 B1 issued Jul. 22, 2003, to Francis J. O'Brien, Jr. et al, published as Publication No. US-2003-0043695-A1 on 6 Mar. 2003, discloses a signal processing system to processes a digital signal converted from to an analog signal, which includes a noise component and possibly also an information component comprising small samples representing four mutually orthogonal items of measurement information representable as a sample point in a symbolic Cartesian four-dimensional spatial reference system. An information processing sub-system receives said digital signal and processes it to extract the information component. A noise likelihood determination sub-system receives the digital signal and generates a random noise assessment of whether or not the digital signal comprises solely random noise, and if not, generates an assessment of degree-of-randomness. The information processing system is illustrated as combat control equipment for undersea warfare, which utilizes a sonar signal produced by a towed linear transducer array, and whose mode operation employs four mutually orthogonal items of measurement information.
The above prior art and prior Department of the Navy development do not disclose a method which utilizes more than one statistical test to decide the structured properties of sparse data in order to maximize the likelihood of a correct decision in processing batches of the sparse data in real time operating submarine systems and/or other contemplated uses.